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ABSTRACT 



.1 statistical model for obtaining a loner 1CC(1 - x)$ 
confidence limit on system reliability vras developed for t"ie 
0e v -rt^ent of Wavy Special projects Office in Pay 19 ' 5 , Tils 
method is a part of NAVJEPS 00 293C-4. Tie accuracy of this 
model is examined by computer simulation for various failure 
ties distributions. The simulation results are presented and 



discussed. 
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CHAPTER I 



INTRODUCTION 

In the last few years reliability has become of primary 
concern in the development of most large weapon systems. This 
growing importance of reliability has produced a great need 
for methods of predicting the overall reliability of these 
systems. Because of the size and complexity of present day 
military weapons system, it is necessary for major contractors 
to have many sub-contractors located throughout the country. 
This fact complicates the problem of formulating a method for 
computing lower confidence limits on system reliability where 
the reliability of the system is computed as the product of 
the reliabilities for the components. 

Recently a system reliability model for computing a 
lower confidence limit for the reliability of weapon systems 
has been developed for the Navy and is contained in "Guide 
Manual for Reliability Measurement Program," NAW/EPS OD 29304, 
15 May 1965 . The model was prepared for the Department of 
Navy Special Projects Office. The document states that the 
model can be utilized by all contractors for subsystem relia- 
bility measurement and by the Navy for weapon system relia- 
bility measurement. The model permits the combination of test 
data from all levels (from component to weapon system) and 
types of tests into meaningful component, equipment, and 
subsystem failure rates and reliability indices. 



? 



An underlying assumption to this statistical model is 
constant failure rate, i.e., the exponential failure law is 



assumed to hold; 

R = e~ X (1) 

where 

R = reliability 
X = failure rate per mission 

It is the purpose of this thesis to examine the accuracy 
of the statistical model contained in OD 29304. The investi- 
gation that follows will examine the model when 

(1) the failure distribution is other than exponential; 
in particular, when failure times have a Rayleigh 
distribution, i.e., increasing failure rate. 
Appendix 1 contains a graph of the Rayleigh distri- 
bution and its failure rate function. 

(2) the failure distribution is a binomial distribution. 

In later chapters , it will be explained that the 
100(1 -a) percentile of the simulated distribution of the 

lower confidence limit, denoted by A n , and the true 

’ 1 - a ’ 

reliability of the system, R , should be equal. Therefore, 

s 

the difference will be used as a measure of the accuracy of 
the model. As an example of some of the results, consider 
the system with 4 components, each having a reliability of 
•995. Samples of sizes of 50 and 500 components were tested 
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with a planned, test time of 2 mission units. The true 

reliability of the system, R , is .9801. The 80th percentile 

s 

points of the distribution of the loi\rer confidence limit for 
Rayleigh distributed failure times were computed to be: 



a 


n i 


A 1 - a | A 1 - a " ^sl 


.20 


50 


.957 .023 


.20 


500 


.961 .019 


,/hen the failure 


times were binomially distributed and the 


test time was one 


mission unit, the 80 th percentile points of 


the distribution 


of the 


lower confidence limit were computed 


to be : 






a 


n i 


A 1 - a | A l-a- B sl 


.20 


50 


.9683 .0120 


.20 


500 


.9800 .0001 



Chapter II contains a complete explanation of the 
statistical model in OD 29304. Chapter III discusses the 
simulation procedures and presents the simulation results. 
It also discusses the two methods used to determine the 
accuracy of the model. Chapter IV discusses and summarizes 
the results and states the conclusions. 



o 



CHAPTER II 



EXPLANATION 0? STATISTICAL MODEL (1) 

If a system consists of k components in logical series, 

the true system reliability, R , may be expressed as 

s 

k 

a = TT h < 2 > 

i = 1 

./here R. is the true reliability of the ith component. 

It is desired to find a. lower 100(1 - o.)% confidence 

A 

limit for R . That is, a statistic R , . such that what- 

S S 9 i_i \ CC ) 

ever the actual values of L, i = l,2,...,k, 

P £ R S ^s,L(aN = (3> 

The proposed method to find this lower 100(1- a )% 

confidence limit for R as outlined in OD 29304 will be pre- 

s 

sented below to include: 

(1) Operating assumptions. 

(2) Estimation of failure rates. 

( 3 ) Statistical equations for confidence limits on 
failure rates and reliability. 

Operating Assumptions 

The reliability measurement system described in OD 29304 
is predicated upon the general reasonableness of the follow- 
ing assumptions: 
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(1) Constant Failure Rate - the exponential failure law 
is assumed to hold. 



(2) Additivity of Stress Effects - the failure rate in- 
duced by two simultaneously acting stresses is 
equal to the sura of the failure rates due to the 
tvro stresses acting sequentially. This assumption 
permits adding data from single environment tests 
together to simulate mission experience. 

(3) Independence of Component Failures - independence 
of component failures is assumed because components 
are normally tested individually by type, and sub- 
system reliability is estimated using component and 
other applicable test results (e.g., equipment and 
subsystem) . 

(4) Failure Rate Constancy - the failure rate is con- 
sidered a function of only the stress acting. In 
other words , the item under test has no memory. 

Estimation of Failure Rates 

Under the above assumptions, if component 1 has failure 
rate , and component 2 has failure rate Xg? then the 
corresponding failure rate for the simple serial subsystem 



Since tne mode of testing generally employed by subsystem 
contractors is to test until some planned test time, the 




(4) 



failure rate estimates, X 



i 



contain bias. Therefore, an 
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unbiased failure rate estimate has been developed in OD 29304 
and is given as 



X. 





2n i + 1 



where 



( 5 ) 



n^^ = sample size for the ith component; i.e., the 
number of tests conducted on component i. 

= the sum of all test times accumulated on the 
n components of type i. 

f = number of components of type i that did not 
complete their mission, i.e., failure time 
was less than the planned test time. 



Under the OD 2930^ method, failure rate estimates for 
subsystem reliability can be calculated from component, 
component-environment, or component-environment-test con- 
dition level data. That is 



A A 

^component “ £ ^component-environment 
A A 

^equipment “ ^ ^component 

A A 

^subsystem ” ^ equipment 



( 6 ) 

( 7 ) 

( 8 ) 



It permits the combination of test data from all levels 
(from component to weapon system) and types of tests into 
meaningful component, equipment, and subsystem failure rates. 
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Statistical Equations 



All statistical estimates, based as they are upon 
fragmentary data, are subject to statistical uncertainty. 

In i eliability work, it is usually of interest to compute 
upper limits on the failure rate and corresponding lower 
limits on the reliability. The following statistical equa- 
tions for calculation of the lower confidence limit are 
based on normal theory and have been corrected to compensate 
for small values of X. Detailed derivations of these 
formulas are contained in reference 1. 

The method proposed is to put n^ items of component 
i on test under the environmental conditions defined in the 
mission and let each operate until failure or the planned 
test time, whichever occurs first. All time is measured in 

mission units. Then an unbiased estimate of fajlure rate, 

A 

, for each component i is obtained from equation (5). 

If failures have occured for any of the i components, an 
upper limit on failure rate can be calculated as follows: 



A 

X 



,24 



= 2X ± (3K) C ± \I4\( i 3K) G ± 



.24 



(BK) 4 $ 2 



U 



(9) 



where 

A k A 



Xl X A 
1=1 


(10) 


A 


ZUj/Si) 

A 


(11) 



K = (l-a)th percentile of the normal distribution 
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3 = correction factor; Table 1 shows appropriate 
beta values for the 8Cth percent confidence 
limit 

■/hen no failures have occurred, the upper limit on failure 
rate is 



A 

X 



(3KP 

k 




JL_ 

S. 

l 



( 12 ) 



wnere 



k = number of components in the simulations 

3 = the sum of all test times accumulated on the 

i 

n. components of type i 



Substitution of the upper limits on the failure rates 
previously obtained xvill generate corresponding lower con- 
fidence limits on reliability; thus : 



A 

SjL(a) 




( 13 ) 



To determine the accuracy of the above model, one needs 



to look at the characteristics of 3 



s ,L(a) 
dence 

system reliability; i.e., from equation (3) 



By definition, 



A 

H , . is a lower 100(1 -a 

s,L(a) 



confidence limit for R , the 

s 



P[R , < R ] = 1 - a 

L s,L(a) - 



Tiis says that R is always the (l-a)th percentile point 

S A 

of the probability distribution of R . . Thus if we 

s , not ) 
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A 

construct the distribution of R , , by computer simulation, 

s > JLAOt ; 

we should find that the (l-a)th percentile point of our 
constructed distribution is R , if equation (3) is correct; 

S 1 

i.e., if in fact R , / , is a true 100(1 -a)# lower confi- 

— s , J- v u. ) 

dence limit for R_. 

s 

Thus a measure for the accuracy of the model is 
I A 1 - a " H s I 

where 



A = (l-a)th percentile point of the distri- 

1 ' a A 

bution of R 

s,L(a) 

If equation (3) is correct, then A, should equal R . 

1 - a s 



TABLE 1 





BETA VALUES 


FOR 80 % 


CONFIDENCE 


LIMIT 




Number of 
Failures 

f i 


Beta 

Values 

*i 


h 


h 


f 

i 


h 


0 


1.507 


10 


1.173 


20 


1.129 


1 


1.369 


11 


1.167 


21 


1.126 


2 


1.309 


12 


l.l6l 


22 


1.124 


3 


1.272 


13 


1.156 


23 


1.122 


4 


1.246 


14 


1.153 


24 


1.119 


5 


1.227 


15 


1.146 


25 


1.117 


6 


1.212 


16 


1.144 


26 


1.115 


7 


1.200 


17 


1.139 


27 


1.113 


8 


1.190 


18 


1.135 


28 


1.111 


9 


1.180 


19 


1.132 


29 


1.110 
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CHAPTER III 



SIMULATION PROCEDURE AND RESULTS 



In order to evaluate the accuracy of the OD 29304 model, 
a computer program was written and used to simulate the 

distribution of The distribution is constructed on 

A 

a computer by generating 500 values of R „ . . for a given 

s ,L(a ) 

set of parameter values k, n(i=l,2,...,k), R^ (i = 1,2, . . . ,k) , 
T oi (i = 1,2,. . . ,k), and a given failure distribution; i.e., 

A 

R , . is a function of 
s,L(a) 



k: number of components in the system 

n^: number of items of component i tested 

R. : reliability of component i 

T q ^ : planned test time for component i 

f^(t): failure distribution for component i 



•.{hen a failure distribution for component i is specified, 
the computer program generates 500 random failure time 
variates, T^ ( j « 1,2, . . . ,500 ) , for component i. If the gener- 
ated time variate is less than the planned test time, T a 
failure is "counted" by the computer. Nith the total number 
of failures, f , counted for a set of 500 generated failure 

times, the computer program then computes the failure rate 

A 

for component i, , from equation (5). The above procedure 

is repeated for each component i, and the corresponding 

A 

upper limit on the system failure rate, a. u , is then calculated 
from equations (9)> (10), and (11). Nith tills value of the 
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system failure rate, the lower confidence limit on the 

A 

system reliability, R , is computed from equation (13). 

s »L(a ) 

The above procedure is replicated, giving 500 values of 
A 

R s These 500 values are then ordered by a separate 

subroutine, and the computer "picks" the (l-a)th percentile 

of these 500 values, denoted by a . This value is then 

compared with the true system reliability R . 

s 

The above simulation procedure was carried out for the 
following two methods : 

(1) Rayleigh distributed failure times 

(2) Binomial distributed failure times 

Rayleigh Distributed Failure Times 

Since the Rayleigh distribution is a one parameter 
distribution (see appendix 1), it x^as decided to choose the 
value of this parameter (a) such that the reliability of a 
component for one mission unit is .995? i.e., 

R^l) = pCTjj >1] = .995 (14) 

The determination of the value of this parameter is shown in 
appendix 2. The resulting failure time distribution that 
will be assumed for each component i in the simulation is: 




f.(t,a) = Ite 2a 2 , a = 10 , t > 0 (15) 

1 or 

For Rayleigh distributed failure times, the accuracy of the 
model was examined for five cases : 
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(1) 


Case 


1: 


T , =0,5 mission units 
ol 


(2) 


Case 


2 : 


T =1.0 mission units 

ol 


(3) 


Case 


3: 


T q1 = 1.5 mission units, 1 = 1,2 








1.35 mission units, 1 = 3*4 


(4) 


Case 


4$ 


T , =2.0 mission units 
oi 


(5) 


Case 


5: 


T q ^ = 5.0 mission units 



Determining the Case 4 planned test time, T o1 , is shown in 

appendix 2. For each case, the number of components k was 

equal to 4, the reliability fi.^ was equal to . 995 » and 

various values of n^ were used. The cases are numbered and 

the results shown in Table 2, The mean and variance of the 

distribution of R„ r/ . are also shown in Table 2. The 

s ,L(a) 

quantity labeled TT in Table 2 is expressed by 

k 

TT = £ n. X (16) 

1 = 1 1 1 



where 



= 1 - = the unreliability of component 1 

This quantity, TT, is a function of the amount of testing 
relative to the unreliability or failure rate. The results 
of the cases in Table 2 imply that the model is more 
accurate as TT increases. 

Binomial Distributed Failure Times 

A 

For this method, the failure rate for component 1, 
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is given as 



A A 

X. = q. = — 
1 1 n i 



( 1 ?) 



where 



f^ = number of components of type i that failed 
n^ = number of items of component i tested 

Using a uniform random number generator, the computer 
generates a uniform random value between 0 and 1 for com- 
ponent i. If the value is greater than R^ , a failure is 
"counted" by the computer. This procedure is repeated n^ 
times for each and for each component i. Thus the num- 
ber of failures counted divided by n^ becomes an estimate 
of the failure rate for component i as given by equation 
(17) above. This process is repeated for all components. 
,/ith these estimates, , the computer simulation determines 

A-, and calculates the mean and variance of the 500 

" a A 

values of R g as previously explained. 

For this method, the accuracy of the model was exam- 



ined for 


three 


cases : 






(1) 


Case 


1: 


R = 
i 


.995 




(2) 


Case 


2: 


R. = 

l 


.950 




(3) 


Case 


3: 


R i = 


.900 




The sum 


of the 


failure 


times, , was set 


equal 




S. = 

l 


•i — 1 


T. . = 
ij 


, i = 1 , 2 ,... 





( 18 ) 
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the number of components k was equal to 4, and various 
values of n. TT ere used. The cases are numbered and the 

i 

results sho T rn in Table 3. Again, the quantity, TT, is 
also listed in Table 3. 
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RESULTS OF COMPUTER SIMULATION 
(Rayleigh Distributed Failure Tiiaes ) 
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RESULTS OF COMPUTER SIMULATION 
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CHAPTER IV 



SUMMARY AND CONCLUSIONS 



It itfas stated previously that a measure of the accuracy 
of the model was given by 



1 - a 



- R 



where A, 




is 


the 


(1- 


a) percentile point of the distri- 


1 - 


■ a 
















but ion of 


A 

R s 


>L(a ) ’ 


and 


R is the 
s 


true reliability of the 


system. 


For 


the Rayleigh distributed failure times 


, this 


measure for 


the 


various 


cases is 


shewn 


below for R 

s 


equal 


to .995. 
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The above 


results bring 


out some 


rather 


interesting: 


point; 


If the planned 


test 


time, T . is 
oi 


large 


, the model 


gives 



pessimistic results; i.e., the accuracy is not very good, 



regardless of the sample size n^. However, if the planned 
test time is small, the accuracy of the model is very good 
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if a large sample of a particular component is tested. in 
other words, for small planned test times, the accuracy 
of the model is very good when the quantity, TT, is greater 
than 6. 

For the Binomially distributed failure times, the 
accuracy of the model for the various cases is given below. 
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The results clearly show that the accuracy of the model is 
extremely good, especially when the sample size, n^, is 
large. 



Obviously, the main reason for the results in the case 
of Rayleigh distributed failure times is the fact that the 
components have increasing failure rates. The underlying 
assumption in this model is that components under test have 
a constant failure rate. In practical applications, many 
components follow this assumption reasonably well, when 
infant mortality failures are deleted and wearout failures 
are not anticipated during the mission duration. However, 
if a new component is being tested, a constant failure rate 
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may not be reasonable at all. Therefore, when this model 
is being used to obtain a lower confidence limit on the 
reliability of a xveapon system, and it is known that a 
particular component has an increasing failure rate, then 
the points brought out above as a result of the simulation 
should be taken into consideration. 
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APPENDIX 1 



RAYLEIGH DISTRIBUTION 



, 1 2a 2 ” 2F 2 

PDF: F(t) = -=£ t e CDF: F(t) - 1-e a 

a 

FR (failure race) z(t) = ~ (increasing failure rate) 



f(t) 




z(t.) 
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APPENDIX 2 



Value of Parameter for Raylelgn Distribution 

The value of the parameter, a, was chosen such that 



1^.(1) = P[T>1] = .995 



(19) 



where 



R.(t) = 1 - F(t) = e 



tr 

2a 2 



( 20 ) 



Equating equations (19) and (20), we obtain 



1 

2a" 



= .995 , t = 1 



( 21 ) 



or 



2 

a = - 



2 (In .995) 



( 22 ) 



hence 



a = 10 



Planned Test Time (T . ) for Case 4- 



The Case 4- planned test time was chosen such that T 

oi 

satisfied the following equation: 



oi 

Z(t)dt = Z(l) 



oi 



(23) 
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where 



Z(t) = -1 t (24) 

a 

This value of T . makes the ’’average" value of Z(t) between 
01 

o and T Qi equal Z(l). Setting t = 1 in equation (24) and 
equating (23) and (24) gives 




or 




hence 




2 . C 



(25) 



( 26 ) 
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APPENDIX 3 



COMPUTER PROGRAM 



DIMENS ION PT ( 4 , 3 ) . RSL ( 500 ) , A ( 3 ) • RSBAR ( 3 ) , SRS ( 3 ) , 

beta(30),rsi(3),hslbar(3)»bs(500).q(4),nm(4) 

READ (5,10) ( NM ( J ) , J = 1,4) , (BETA(K) ,K = 1 , 30 ) , ( Q( I ) , I = 1 ) 
10 FORMAT ( 414/16 J?5 . 3/14F5 . 3/4f6 . 3 ) 

TPR = URN ( 0 ) 

DO 999 K = 1,4 

SS = NM ( K ) 

SRS (K) = 0.0 
A(K) = o.o 
RSLBAR(K) =0.0 
SRSL(K) = 0.0 
F2 = NM(K) 

B = (2.0 * F2 )/ (2 s 0 * F2) + 1.0) 

This loop generates 500 time variates for each component 

DO 900 M = 1,500 
FS = 0.0 
VU = 0.0 
C = 0.0 
VI = 0.0 
RSL(M) =0.0 
RS ( M ) =0.0 
DO 800 I = 1,4 
FF = 0.0 
VL = 0.0 
NL = NM ( K ) 

DO 700 J = 1 , NL 
TPR = URN(1 ) 

IF(TPR.GT. 3(1) )GO TO 605 
GO TO 700 
605 FF = FF + 1.0 
700 Continue 
VL = FF/F2 
VI = VI + VL 
FS = FS + FF 
C = C + (VL/SS) 

VU = VU + (l.O/'SS) 

800 CONTINUE 

A -J 

This statement computes a value of R g = e 

RS ( M ) = EXP ( -VI ) 

RSBAR(K) = RSBAR (K) + RS(M) 

1F( FS ,GT. 0.0 )GO TO 842 
BTA = BETA(l) 

VU = (VU*BTA)*(BTA*.17724l) 

Go to 832 
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842 C = C/Vl 

IP (PS .GE. 30.0)GO TO 820 
LI = PS + 1.0 
BTA = BETA (LI) 

GO TO 825 
820 BTA =1.0 

825 B2 = (BTA * .708964) * BTA 

D = ((4.0 * V1)*(B2*C)) + ((B2*B2)*(C*C)) 

D = S^RT(D) 

VU = (((2.0 * VI) -i- (B2*C)) + D)/(2. 0 ) A 

a 

This statement computes a value of R s = e 



832 RSL(M) = EXP(-VU) 

RSLBAR(K) = RSLBAR(K) + RSL(M) 

900 CONTINUE 

The following statements compute the mean and variance of 
A A 

R_ and R 
s s,L 

RSBAR(K) = R3BAR(K)/500.0 
RBAR = RSBAR(K) 

DO 905 M = 1,500 

SRS(K) = SRS(K) + (RS(M) - RBAR)**2 
905 CONTINUE 

SRS(K) = SQHT(SRS(K)/500.0) 

RSLBAR(K) = RSLBAR(K)/500.0 
RLBAR = RSLBAR(K) 

DO 2410 J = 1,500 

SRSL(K) = SRSL(K) + (RSL(J) - RSBAR)**2 
2410 CONTINUE 

SRSL(K) = S^RT(SRSL(K)/5C0.0) 

/\ 

The following statements order the 500 values of R , and 

picks the 80th percentile of the 500 values, A . ’ 

X — cc 

II = 1 

DO 9ic I = 1,101 
TEMP = RSL(l) 

DO 915 j = 1,500 
IF (TEMP ~ RSL(J) )9l6, 915,915 
916 TEMP = RSL(J) 

II = J 

915 CONTINUE 

RSL(II) = 0.0 
A(K) = TEMP 
9.0 CONTINUE 

The following statements print out the results- A-, , mean 

and standard deviation of fi c , , and the mean and standard 
deviation of R . 
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rffiITE(6,1010 ) A(K) ,hslbar(k) ,srsl(k) ,rsbar(k) ,SRS(K) 
1010 FORMAT (//o FI 3. 5) 

999 CONTINUE 
END 
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